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What GAO Found 


The federal investment in science, technology, engineering, and 
mathematics (STEM) education programs remained relatively stable from 
fiscal years 2010 to 2016, although the number of programs declined 
from 209 to 163 (see figure). While agencies reported that many of the 
same STEM education programs existed during this time period, the 
portfolio underwent various changes, including program consolidations, 
creations, and terminations. Nearly all STEM education programs in fiscal 
year 2016 overlapped to some degree with at least one other program in 
that they offered similar services to similar groups in similar STEM fields 
to achieve similar objectives. The Committee on STEM Education, an 
interagency body responsible for implementing the federal STEM 
education strategic plan, reported it managed this overlap through 
coordination with agencies administering these programs. 


Number of Federal Science, Technology, Engineering, and Mathematics (STEM) Education 
Programs Reported in Fiscal Years 2010 and 2016 


2010 2016 


54 
new 


209 


programs 


163 


programs 


education programs declined 


$3.1 Billion $2.9 Billion 


Source: GAO analysis of information reported by agency officials on STEM education programs. | GAO-18-290 


The Committee on STEM Education has not fully met its responsibilities 
to assess the federal STEM education portfolio. Specifically, the 
Committee has not reviewed programs’ performance assessments, as 
required by its authorizing charter, nor has it documented those 
assessments in its inventory, as required by law. Such efforts could 
encourage the use of evidenced-based practices across the portfolio—a 
key national goal of the STEM education strategic plan. These efforts 
could also enhance public awareness of the administering agencies’ 
efforts to assess programs’ performance. In addition, the Committee has 
not reported the participation rates of underrepresented groups in federal 
STEM education programs, as required by law. By reporting this 
information, the Committee could better assess whether programs are 
broadening access to groups historically underrepresented in STEM 
fields—another key goal of the strategic plan. 
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House of Representatives 


The Honorable Robert C. Scott 

Ranking Member 

Committee on Education and the Workforce 
House of Representatives 


Education programs in science, technology, engineering, and 
mathematics (STEM) play an important role by preparing students for 
careers in STEM fields and are intended to enhance the nation’s global 
competitiveness. Various levels of government foster STEM education, as 
do other stakeholders, including institutions of higher education, 
professional and scientific societies, and private industries. In 2012, we 
reported that more than 200 federal STEM education programs across 13 
different agencies spent more than $3 billion in fiscal year 2010.’ We also 
found a high degree of overlap among those programs and made several 
recommendations to improve their coordination and management. You 
asked us to review the current landscape of federal STEM education 
programs. This report examines: 1) how the federal investment in STEM 
education programs changed from 2010 to 2016, and 2) to what extent 
the STEM education portfolio has been assessed. 


To gather information on how the federal investment in STEM education 
changed from 2010 to 2016, we administered a web-based questionnaire 
to STEM education programs funded in fiscal year 2016. We identified 
these programs based on input from the 13 agencies that administered 
the STEM education programs we previously reported for fiscal year 
2010. Program officials provided information on all 163 programs funded 


1 GAO, Science, Technology, Engineering, And Mathematics Education: Strategic 
Planning Needed to Better Manage Overlapping Programs across Multiple Agencies, 
GAO-12-108 (Washington, D.C.: Jan. 20, 2012). 


2 GAO-12-108 
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in fiscal year 2016 that we determined met our definition of a STEM 
education program. For more information on our definition, program 
identification, and data collection, see appendix |. The questionnaire 
solicited information on the programs’ objectives, groups served, services 
provided, and spending, among other things, and we analyzed program 
officials’ responses. We used this information to gauge the level of 
overlap among programs funded in fiscal year 2016. We also compared 
the results to similar data previously collected for programs funded in 
fiscal year 2010. 


To understand the extent to which the federal programs within the STEM 
education portfolio had been assessed, we reviewed the requirements in 
the America COMPETES Reauthorization Act of 2010 (COMPETES Act)? 
regarding federal STEM education program assessment.* Subsequently, 
we examined the information produced to fulfill these requirements, 
including a strategic plan and annual progress reports. We sent our initial 
determination of whether these COMPETES Act requirements had been 
fulfilled to the entities responsible for implementing them and solicited 
their views. We obtained a response from the Office of Science and 
Technology Policy as to whether the requirements had been fulfilled and 
considered any additional information the agencies provided in finalizing 
our analysis. In addition, we reviewed the assessment responsibilities of 
the Committee on STEM Education as specified in its authorizing charter 
and examined its efforts to meet those responsibilities. Further, we 
conducted a limited review to test whether program assessment 


3 Pub. L. No. 111-358, 124 Stat. 3982. 


4 For the purposes of our report, we use the term performance assessment to refer to 
agencies’ efforts to assess the effectiveness of their programs. We have previously 
reported that performance assessment is an important way to obtain evaluative 
information that can help demonstrate whether and why a program is working well. 
Agencies may use different methods to assess the performance of their programs. For 
example, performance measurement focuses on whether a program has achieved its 
objectives, expressed as measureable standards, while program evaluations typically 
examine a broader range of information on program performance and its context. See, 
GAO, Managing for Results: Enhancing Agency Use of Performance Information for 
Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9, 2005); 
Performance Measurement and Evaluation: Definitions and Relationships, 
GAO-11-646SP (Washington, D.C.: May 2011); and Program Evaluation: Strategies to 
Facilitate Agencies’ Use of Evaluation in Program Management and Policy Making, 
GAO-13-570 (Washington, D.C.: June 26, 2013). 
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information was routinely contained within agency budget documents.° In 
response to our questionnaire, program officials reported that 73 of the 
163 STEM education programs had been evaluated since fiscal year 
2010. From those 73, we randomly selected a nongeneralizable sample 
of 10 STEM education programs—one from each agency that reported it 
had evaluated at least one of its STEM education programs recently. We 
then searched for supporting information within the agencies’ budget 
documents. 


In addition, we interviewed officials responsible for coordinating federal 
STEM education efforts from the National Science Foundation and the 
Office of Science and Technology Policy. We also interviewed officials 
from the Office of Management and Budget and other federal agencies 
leading strategic interagency efforts on STEM education, including the 
National Aeronautics and Space Administration, the Departments of 
Education, Energy, Health and Human Services, and the Smithsonian 
Institution. Further, to inform both of our objectives, we reviewed relevant 
federal laws and regulations as well as reports on STEM education. 


We conducted this performance audit from October 2016 to March 2018 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


a —————————_— eee 


Background 


STEM Education The term “STEM education’ includes educational activities across all 
grade levels—from preschool to graduate school. STEM education 
programs have a variety of primary objectives, which include preparing 
students for STEM coursework, providing postsecondary students with 
grants or fellowships in STEM fields, and improving STEM teacher 
training (see appendix | for our definition of STEM education programs).°® 


5 For this review, we used the definition of performance assessment found in 
GAO-11-646SP. Therefore, if an entry referenced an evaluation or contained performance 
information, we included it in our total. 


8 This definition is consistent with the definition of STEM education used in GAO-12-108. 
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GAO’s Work on Duplication, Overlap, 
and Fragmentation 


This report, like GAO’s prior work, uses 
standard definitions to describe duplication, 
overlap, and fragmentation, among 
government programs: 


° Duplication occurs when two or more 
agencies or programs are engaged in 
the same activities or provide the same 
services to the same beneficiaries. 


e Overlap occurs when multiple agencies 
or programs have similar goals, engage 
in similar activities or strategies to 
achieve their goals, or aim to serve 
similar beneficiaries. 


e Fragmentation refers to those 
circumstances in which more than 
one federal agency (or more than one 
organization within an agency) is 
involved in the same broad area of 
national need and opportunities exist 
to improve service delivery. 

Source: GAO, Fragmentation, Overlap, and Duplication: An 


Evaluation and Management Guide, GAO-15-49SP 
(Washington, D.C.: April 14, 2015). | GAO-18-290 


America COMPETES 
Reauthorization Act of 
2010 


Federal STEM education programs have been created in two ways— 
either by law or by federal agencies under their statutory authorities. 


We previously reported that most federal STEM education programs 
overlapped to some degree with at least one other program, in that they 
offered similar services to similar groups in similar STEM fields to achieve 
similar objectives (see sidebar for definition of overlap).’ 


Although those programs may not be duplicative, we reported that they 
were similar enough that they needed to be well coordinated and guided 
by a robust strategic plan. And, through its strategic planning and other 
coordination efforts, the Office of Science and Technology Policy and the 
National Science and Technology Council implemented our 
recommendations to work with agencies to better align their activities with 
a government-wide strategy; develop a plan for sustained coordination; 
identify programs for potential consolidation or elimination; and assist 
agencies in determining how to better evaluate their programs.® 


Enacted in 2007, the America COMPETES Act authorized several 
programs to promote STEM education.? The America COMPETES 
Reauthorization Act of 2010 (COMPETES Act) reauthorized the America 
COMPETES Act and addresses coordination and oversight issues, 
including those associated with the coordination and potential duplication 
of federal STEM education efforts.'° The COMPETES Act required the 
Director of the Office of Science and Technology Policy to establish, 
under the National Science and Technology Council, the Committee on 


1 GAOA2408. 


8 The National Science and Technology Council is the principal body within the executive 
branch that coordinates science and technology policy across the federal government's 
research and development efforts. 


° Pub. L. No. 110-69, 121 Stat. 572 (2007). 


1° Bub. L. No. 111-358, 124 Stat. 3982. For ease of reference, for purposes of this report 
we refer to the America COMPETES Reauthorization Act of 2010 as the COMPETES Act. 
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STEM Education to serve as the interagency coordination body for STEM 
education in the federal government (see fig. 1)." 


aaa er ee ee | 
Figure 1: Leadership Structure for Policy on Science, Technology, Engineering, and 
Mathematics (STEM) Education 


Office of Science and 
Le Technology Policy 
W Advises the president on 


the effects of science and 
technology on domestic 
and international affairs 


nr 
National Science and 
Technology Council 
Coordinates science 
and technology policy 
with federal government's 


larger research and 
development efforts 


Kee. 


* 


USDA 
=z 
The Committee on STEM Education 

Coordinates and assesses federal programs and activities across 
federal agencies in support of STEM education 


Leading agencies include the Office of Science and Technology 
Policy and the National Science Foundation 


Source: GAO. | GAO-18-290 


In May 2013, the Committee on STEM Education issued a 5-year 
Strategic Plan for federal STEM education efforts, as required by the 
COMPETES Act. To improve collaboration across the portfolio, the 
Strategic Plan identified five priority investment areas and two 
coordination objectives, specifying national goals for each (see fig. 2). 


"1 Bub. L. No. 111-358, § 101(a), 124 Stat. 3982, 3984 (codified at 42 U.S.C. § 6621(a)). 
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Figure 2: Science, Technology, Engineering, and Mathematics (STEM) Education National Goals, Priority Investment Areas, 
and Coordination Objectives 


National goals 5 priority areas 2 objectives National goals 
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Increase Build new the public 

and sustain models for 

youth and public leveraging 
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Support a 50 Design A fehl 
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8. experience of serve groups education for to achieve the most 
youth who have undergraduate historically tomorrow’s 


significant impact from 
students under- workforce ‘ federal STEM education 


represented investments 
in STEM 
fields 


an authentic STEM 
experience each year 
prior to completing 
high school 


Provide 
graduate-trained 
STEM professionals with 


Graduate one million research expertise and 


additional students with Increase the number specialized skills for relevant 
degrees in STEM fields of graduates from federal agencies, and 
over the next 10 years underrepresented groups other skills for career 

in STEM fields in the next success 


10 years and improve 
women’s participation in 
underrepresented areas 


Source: 5-year STEM Education Strategic Plan. | GAO-18-290 


Note: The Strategic Plan defines historically underrepresented groups as Hispanics and Latinos, 
African Americans, American Indians, Alaska Natives, Native Hawaiians and Pacific Islanders, the 
economically disadvantaged, people with disabilities, and women and girls. 


The COMPETES Act also requires that the Committee create, and 
periodically update, an inventory of federal STEM education programs 
that includes documentation of program assessments and the 
participation rates of women, underrepresented minorities, and persons in 
rural areas. '2 


12 49 U.S.C. § 6621(b)(6). 
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In addition, the COMPETES Act requires that the Office of Science and 
Technology Policy publish annual reports on coordinating federal STEM 
education efforts. The law mandates that these reports include specific 

information, such as: 


e adescription of each federal agency’s STEM education programs 
funded in the previous and current fiscal years, as well as those 
proposed under the President’s budget request; 


e the levels of funding for each participating federal agency’s programs 
described above; 


e an evaluation of the levels of duplication and fragmentation of the 
programs described above; and 


e adescription of the progress made implementing the Strategic Plan, 
including a description of the outcome of any program assessments 
completed in the previous year, and any changes made to the 
Strategic Plan since the previous annual report. '° 


In January 2017, the President signed into law the American Innovation 
and Competitiveness Act, which, among other things, amended certain 
provisions of the COMPETES Act."* The Act added some requirements 
for both the Office of Science and Technology Policy and the Committee 
on STEM Education. For example, it created new mandates for the 
Committee to: 


e review the measures federal agencies use to evaluate their STEM 
education programs, and 


e make recommendations for reforming, terminating, or consolidating 
the federal STEM portfolio. ' 


13 49 U.S.C. § 6621(d). 
14 Bub. L. No. 114-329, 130 Stat. 2969 (2017). 


15 pub. L. No. 114-329, § 304(a)(3), 130 Stat. 2969, 3006 (2017) (codified at 42 U.S.C. § 
6621(b)(8), (10)). 
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Cross-agency Priority 
Goal on STEM Education 


Other Data and 


Any such recommendations for an upcoming fiscal year are to be 
included in the Office of Science and Technology Policy's annual report. '® 


In 2014, the Office of Management and Budget, in consultation with the 
federal agencies that administer STEM education programs, established 
STEM education as a cross-agency priority goal.'” The Office of Science 
and Technology Policy and the National Science Foundation led the 
oversight and management of this goal, and as part of this work, goal 
leaders from these agencies identified milestones that aligned with the 
Strategic Plan’s priority investment areas and coordination objectives 
(see fig. 2). For example, goal leaders reported progress toward meeting 
key milestones associated with improving STEM instruction. In 2017, to 
ensure alignment with the current administration’s priorities, the Office of 
Management and Budget removed the priority status of all cross-agency 
priority goals, including STEM education; this ended the required public 
issuance of quarterly priority goal reports. '® The STEM Education goal’s 
final quarterly progress report was issued at the end of fiscal year 2016. 


Other government-wide efforts are underway to improve the transparency 


Transparency 
Requirements 


around federal programs in general. These efforts are not directed at the 
STEM education programs specifically, but may assist in managing the 
STEM education portfolio. The GPRA Modernization Act of 2010 requires 
the Office of Management and Budget to present a coherent inventory of 
all federal programs by making information about each federal program 
available on a website. '? However, we previously reported that, because 
agencies used different approaches to define their programs, 
comparability of programs within and across agencies on this inventory 


16 The Office of Science and Technology Policy’s annual report is mandated to be sent to 
Congress at the time of the President’s budget request, which is required by law to be 
submitted no later than the first Monday in February. 42 U.S.C. § 6621(c), 31 U.S.C. § 
1105(a). 


wv Cross-agency priority goals are 4-year outcome-oriented goals covering a number of 
complex or high-risk management and mission issues. 


18 GAO, Managing for Results: Further Progress Made in Implementing the GPRA 
Modernization Act, but Additional Actions Needed to Address Pressing Governance 
Challenges, GAO-17-775 (Washington, D.C.: Sept. 29, 2017). 


19 Pub. L. No. 111-352, § 7, 124 Stat. 3866, 3876 (2011), codified at 31 U.S.C. § 1122. 
The GPRA Modernization Act of 2010 enhanced the Government Performance and 
Results Act of 1993 (GPRA) (Pub. L. No. 103-62, 107 Stat. 285 (1993)). 
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was limited.2° We recently identified a potential framework for the 
development of a useful federal program inventory.2' The Office of 
Management and Budget decided to postpone further development of the 
inventory in order to coordinate with the implementation of related 
requirements of the Digital Accountability and Transparency Act of 
2014.2 Once fully implemented, this act is expected to expand the types 
and transparency of public information on federal spending to make it 
easier to track it to specific federal programs. The act requires 
government-wide reporting on a greater variety of data related to federal 
spending, such as budget and financial information, as well as tracking of 
these data at multiple points in the federal spending lifecycle.?° 


From 2010 to 2016, 
the Number of STEM 
Education Programs 
Decreased While 
Spending Remained 
Stable, and Most 
Programs Continued 
to Overlap 


20 GAO, Government Efficiency and Effectiveness: Inconsistent Definitions and 
Information Limit the Usefulness of Federal Program Inventories, GAO-15-83, 
(Washington D.C.: Oct. 31, 2014). 


21 GAO, Federal Programs: Information Architecture Offers a Potential Approach for 
Development of an Inventory, GAO-17-739 (Washington, D.C.: Sept. 28, 2017). 


22 Bub. L. No. 113-101, 128 Stat. 1146. 


3 We recently reported on implementation of the Act. GAO, DATA Act: OMB, Treasury, 
and Agencies Need to Improve Completeness and Accuracy of Spending Data and 
Disclose Limitations, GAO-18-138 (Washington, D.C.: Nov. 8, 2017). 
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Agencies Reported Fewer 
STEM Education 
Programs and Relatively 
Stable Levels of Spending 
in Fiscal Year 2016 
Compared to Fiscal Year 
2010 


Program officials from the 13 federal agencies that administer STEM 
education programs reported a total of 163 STEM education programs in 
fiscal year 2016, compared to 209 programs in fiscal year 2010. Three 
agencies—the Department of Energy, the Department of Health and 
Human Services, and the National Science Foundation—administered 
more than half of all STEM education programs in fiscal years 2010 and 
2016. 


Despite collectively reporting fewer STEM education programs, program 
officials responding to our questionnaire reported spending about the 
same amount in fiscal year 2016 as they did in fiscal year 2010. In fiscal 
year 2016, program officials reported spending about $2.9 billion on the 
163 programs.** Spending by individual programs ranged from about 
$14,000 annually to hundreds of millions of dollars.2° The National 
Science Foundation and the Department of Health and Human Services 
programs account for about 60 percent of this spending. Figure 3 
provides an agency-level summary of the number of programs and their 
reported spending. Appendix II contains a complete list of the 163 STEM 
education programs and their reported spending for fiscal year 2016.7° 


24 Fiscal year 2010 obligations for STEM education programs were $3.1 billion. When 
adjusted for inflation, these program obligations are approximately $3.4 billion in 2016 
dollars. 


*5 The Department of Agriculture’s 1890 National Scholars Program has not obligated any 
funds to the public. It is a share-cost program supported by Department of Agriculture 
agencies. 


26 Federal agencies also carried out other activities related to STEM education; however, 
as we previously reported, these activities did not fit our definition of a STEM education 
program because STEM education was not the primary objective. These efforts include 
broad-based programs with STEM components, programs that enhance the general 
public’s knowledge of STEM, and research programs that may hire students. For example, 
grant programs may offer competitive preference to applicants that propose activities in 
alignment with the administration’s priority of advancing STEM education, without making 
it a requirement for funding. 
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Figure 3: Number of Science, Technology, Engineering, and Mathematics (STEM) Education Programs by Agency and Total 


Reported Spending, Fiscal Year 2016 


Obligations in billions (FY2016) 
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Source: GAO analysis of fiscal year 2016 data as reported by department and agency officials. | GAO-18-290 NRC: Nuclear Regulatory Commission 


Note: GAO asked program officials to report on obligations—defined as definite commitments that 
create a legal liability of the government for the payment of goods and services ordered or received, 
or a legal duty on the part of the United States that could mature into a legal liability. Payment may be 
made immediately or in the future. An agency incurs an obligation, for example, when it places an 
order, signs a contract, awards a grant, purchases a service, or takes other actions that require the 
government to make payments to the public or from one government account to another. In this 
report, we generally refer to obligations as spending. 


While agencies reported many of the same STEM education programs in 
fiscal years 2010 and 2016, the federal portfolio evolved in various ways. 
About half of the 209 programs previously reported for fiscal year 2010 
were reported again for fiscal year 2016—accounting for about two-thirds 
(109 programs) of the fiscal year 2016 portfolio. The remaining third (54 
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programs) were newly reported for fiscal year 2016. (See appendix | for 
more information on changes to the STEM portfolio between fiscal years 
2010 and 2016.) 


The portfolio underwent various changes from fiscal years 2010 to 2016, 
including program consolidations, creations, and terminations.?’ 
According to leadership of the Committee on STEM Education, these 
changes were due to many factors. One key factor is the STEM 
Education Strategic Plan, which, among other things, calls for greater 
efficiency and cohesion across federal STEM education programs. Other 
factors include agencies’ individual priorities, including their mission and 
budget, and congressional interest in specific programs. For example, 
agencies reported: 


e Consolidations. Starting in 2014, for greater efficiency and cohesion, 
the National Science Foundation consolidated a number of related 
undergraduate STEM education programs, including STEM Talent 
Expansion Programs, Transforming Undergrad Education in STEM, 
and Nanotechnology Undergraduate Education in Engineering. 


e Creations. Department of Health and Human Services officials 
reported administering 28 new STEM education programs. These 
programs are housed in the Department’s National Institutes of 
Health, which generally bases its funding decisions on scientific 
opportunities and its own peer review process. One new program is 
the Building Infrastructure Leading to Diversity Initiative. This program 
supports undergraduate institutions in implementing and studying 
approaches to engaging and retaining students from diverse 
backgrounds in biomedical research. 


e Terminations. Department of Education officials reported that four 
STEM education programs funded in fiscal year 2010 were terminated 
before fiscal year 2016. One such program was the Women’s 
Educational Equity program. Congress last funded this program in 
fiscal year 2010. 


27 In addition, 20 programs previously identified as STEM education for fiscal year 2010 
were not recognized by their administering agency as STEM education programs in fiscal 
year 2016. (See appendix | for more information about the criteria and methodology we 
used to exclude a program from our inventory.) 
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Significant Overlap Based on our analysis of questionnaire responses, nearly all STEM 

Continued to Exist Among education programs in fiscal year 2016 overlapped with at least one other 

STEM Education STEM education program, in that they offered at least one similar service 
to at least one similar group in at least one similar STEM field to achieve 

Programs, Although at least one similar objective (see text box). Similar levels of overlap 

Programs May Differ in occurred among programs funded in fiscal year 2010. 

Meaningful Ways 


Similarities Among Overlapping Federal Science, Technology, Engineering, and Mathematics (STEM) Education Programs 


Similar Services 


Many of the 163 STEM education programs provided similar services. To support students, most programs (143) provided research 
opportunities, internships, mentorships, or career guidance. In addition, 110 programs supported short-term experiential learning 
activities, and 99 programs supported long-term experiential learning activities. Short-term experiential learning activities include field 
trips, guest speakers, workshops, and summer camps. Long-term experiential learning activities last throughout a semester in length 
or longer. To support teachers, 77 programs provided curriculum development and 45 programs supported teacher in-service training, 
professional development, or retention activities. 


Similar Groups Intended to be Served 


Many programs also provided services to similar groups, such as K-12 students, postsecondary students, K-12 teachers, and college 
faculty. A majority of STEM programs reported primarily benefiting postsecondary students; specifically, 103 programs intended to 
serve 4-year undergraduate students, 76 intended to serve Master’s degree students, and 83 intended to serve doctoral students. 
Most programs also intended to serve multiple groups; 137 of the 163 programs served two or more groups. 


Similar STEM Fields 


More than 75 percent of programs focused on specific STEM academic fields of study. The most common fields were biology (85 
programs), technology (75 programs), engineering (72 programs), and computer science (71 programs). Of those programs that 
focused on specific STEM fields of study, about 55 percent (68 programs) focused on 5 or more different fields. 


Similar Objectives 


Many STEM education programs had similar objectives. An objective of a majority of programs (115) was to provide training 
opportunities for undergraduate or graduate students in STEM fields. Most programs (139) also reported having multiple primary 
STEM objectives. 


Source: GAO analysis of questionnaire responses provided by agency officials on STEM education programs.| GAO-18-290 


Despite these similarities, overlapping programs may differ in meaningful 
ways, such as their specific field of focus and those programs’ stated 
goals. For example, a primary objective of the Department of Health and 
Human Services’ Cancer Education Grants program and the National 
Aeronautics and Space Administration’s National Space Grant College 
and Fellowship Project is to provide training opportunities for 
undergraduate or graduate students in biological sciences, among other 
fields. However, these programs have different program goals: 
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e The Cancer Education Grants program aims to develop innovative 
cancer education programs and cancer research dissemination 
projects. 


e The National Space Grant College and Fellowship Project encourages 
interdisciplinary education, research, and public service programs 
related to aerospace. 


Although many STEM education programs are designed to provide 
similar services to similar groups, some programs serve distinct 
populations within those broader groups, such as minority, 
disadvantaged, or underrepresented groups.”® Within the broad group— 
middle and high school students, an individual program may focus on 
serving only minority, disadvantaged, or underrepresented students. For 
example, the Department of Transportation’s Garrett A. Morgan 
Technology and Transportation Education program focuses services on 
students who are girls and minorities, whereas the Department of 
Education’s Upward Bound Math-Science program aims to serve 
students who are economically disadvantaged. 


The Committee on STEM Education and the Office of Science and 
Technology Policy reported managing overlap in the portfolio by 
coordinating with other agencies through a: 


e Cross-agency priority goal. Project management and oversight of 
this goal provided an additional mechanism to facilitate coordination. 
Goal leaders published quarterly progress reports describing their 
efforts to achieve each of the five priority investment areas and two 
coordination objectives.”° 


e Federal coordination subcommittee. Creating a federal 
coordination subcommittee and various interagency working groups 
helped to advance goals identified in the Strategic Plan. Committee 
leadership structured working groups to connect agencies with similar 
programs (see fig. 4). 


28 in response to our questionnaire, program officials reported that 64 of the 163 (39 
percent) programs administered in fiscal year 2016 were primarily intended to serve 
minority, disadvantaged, or underrepresented groups. 


2° GAO has previously reported that the governance structure around cross-agency 
priority goals has been effective in improving coordination around those goals. GAO, 
Managing for Results: OMB Improved Implementation of Cross-Agency Priority Goals, but 
Could Be More Transparent About Measuring Progress, GAO-16-509 (Washington, D.C.: 
May 20, 2016). 
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Figure 4: Federal Oversight and Working Groups on Science, Technology, 
Engineering, and Mathematics (STEM) Education 
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Source: GAO analysis of information reported by agency officials on STEM education programs. | GAO-18-290 


Note: The cross-agency priority goal on STEM education ended in May 2017. Also, in August of 
2016, subsequent to the issuance of the Strategic Plan, the Committee on STEM Education initiated 
the Computer Science for All working group. 
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Efforts to Assess 
Programs’ 
Performance and 
Participation Rates of 
Underrepresented 
Minorities Are Limited 
Performance Assessments 
of STEM Education 


Programs Are Not 
Reviewed or Documented 


The Committee on STEM Education and Office of Science and 
Technology Policy have not fully met their responsibilities to assess the 
STEM education portfolio.*° Specifically, the Committee on STEM 
Education has not reviewed performance assessments of STEM 
education programs to ensure effectiveness—a primary function of its 
authorizing charter.*1 Committee leadership acknowledged that they have 
not conducted such reviews. Overall, the Committee made limited 
progress advancing its strategic goal of increasing the use of evidence- 
based approaches because, according to Committee leadership, they 
focused on achieving other strategic goals. By reviewing programs’ 
performance assessments, the Committee could leverage existing 
performance information to identify and share promising practices that 
agencies could use in designing or revising their programs. Moreover, in 
doing so, the Committee could further its strategic goal of increasing the 
use of evidence-based approaches across the portfolio of STEM 
education programs. We previously have reported that managers can use 
performance information to identify and increase the use of program 
approaches that are working well.°? Additionally, such a review could help 
the Committee meet its new responsibilities under the 2017 American 
Innovation and Competitiveness Act, including reviewing the measures 
federal agencies use to evaluate their STEM education programs and 
making recommendations for terminating, consolidating, and reforming 
programs in the federal STEM education portfolio. 


30 See appendix III for the implementation status of selected COMPETES Act provisions. 


31 In 2011, the National Science and Technology Council issued a charter establishing the 
Committee on STEM Education. The charter describes the Committee’s purpose and 
scope, functions, and membership. One of its three functions is to review STEM education 
activities and programs, and the respective assessments of each, throughout federal 
agencies to ensure effectiveness. 


32 GAO, Managing for Results: Enhancing Agency Use of Performance Information for 
Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9, 2005). 
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Further, the Committee on STEM Education has not met the COMPETES 


COMPETES Act Requirement of the 

Committee on Science, Technology, 

Engineering, and Mathematics Education 

e Create, periodically update, and maintain 
an inventory of federal STEM education 
programs that includes documentation 
of program assessments. 

Source: 42 U.S.C. § 6621(b)(6). | GAO-18-290 


COMPETES Act Requirement of the Office 

of Science and Technology Policy 

° Include in its annual report a description 
of any program assessments completed 
in the previous year. 

Source: 42 U.S.C. § 6621(d)(4). | GAO-18-290 


Act requirement to document the performance assessments of STEM 
education programs in its federal STEM inventory (see sidebar). 


In 2011, the Committee on STEM Education reported summary 
information on programs’ performance assessments, including the total 
number of programs funded in fiscal year 2010 that had been evaluated 
since 2005.3 However, the information provided was not program- 
specific; therefore, it is unclear which programs were assessed for 
effectiveness. Further, that information is outdated, as the STEM 
education portfolio has changed considerably since 2010, as we have 
discussed in this report. Committee leadership said they do not have 
plans to update the summary information provided in 2011, noting that 
agency budget justifications include program performance assessments. 
However, we reviewed the budget justifications for 10 STEM education 
programs that program officials reported had been recently evaluated and 
found that 8 had no information on performance assessments. By 
periodically documenting in its federal STEM education inventory whether 
programs have been assessed for effectiveness, the Committee can 
enhance communication of performance information among agency 
officials and stakeholders. This could facilitate the use of performance 
information by agency managers and lead to greater public awareness 
regarding the effectiveness of many of the nation’s STEM education 
programs. 


The Office of Science and Technology Policy has not done everything 
required of it either. It has not described the outcomes of programs’ 
performance assessments completed in the previous year in its annual 
reports, as required by the COMPETES Act (see sidebar). 


2 According to The Federal Science, Technology, Engineering, and Mathematics (STEM) 
Education Portfolio, the 252 STEM education investments reported in fiscal year 2010 
were categorized as either broader STEM education or agency-mission specific. Of the 
139 broader STEM education investments, 119 had been evaluated since 2005. The 113 
agency-mission specific investments were less thoroughly evaluated; 46 of the 113 had 
collected some outcome data. 
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Program Participation 
Rates of 
Underrepresented Groups 
Are Not Reported 


COMPETES Act Requirement of the 
Committee on Science, Technology, 
Engineering, and Mathematics Education 


e Create, periodically update, and maintain 
an inventory of federal STEM education 
programs that includes documentation of 
participation rates of women, 
underrepresented minorities, and persons 
in rural areas. 

Source: 42 U.S.C. § 6621(b)(6). | GAO-18-290 


Office of Science and Technology Policy officials said that they have not 
reported on recent program assessments, and added that many STEM 
education programs were not mature enough to provide sufficient data for 
a definitive assessment. However, many of the 2016 programs that we 
identified were at least 7 years old and had been assessed. Specifically, 
67 percent (109) of the programs reported by program officials for fiscal 
year 2016 had also been reported for fiscal year 2010. Of the programs in 
existence since 2010, 49 percent (53) have been assessed, according to 
program officials’ questionnaire responses. By reporting information on 
the outcomes of performance assessments completed in the previous 
year, the Office of Science and Technology Policy could enhance 
awareness of promising practices in federal STEM education programs. 


The Committee on STEM Education has not reported STEM education 
programs’ participation rates of groups historically underrepresented in 
STEM fields, although broadening participation of those groups is one of 
the Committee’s strategic goals.** Moreover, the COMPETES Act 
requires that the Committee report the participation rates of women, 
underrepresented minorities, and persons in rural areas in its inventory of 
federal programs (see sidebar). 


Committee leadership acknowledged they have not reported these data, 
and added that such participation data are not fully available across all 
STEM education programs. However, we found that such participation 
data were generally available. 


In response to our questionnaire, nearly three-quarters of STEM 
education programs (120 of 163) reported tracking participants in fiscal 
year 2016. Of those programs, many also tracked specific participant 
characteristics. For example, 61 percent (73) of programs that tracked 
participants also captured whether their participants were women and 54 
percent (65) documented those who were African American. Programs 
primarily intended to serve minority, disadvantaged, or underrepresented 
groups tracked participant characteristics at higher rates than programs 
that intended to serve broader groups of beneficiaries (see fig. 5). 


34 One priority investment area specified in the Strategic Plan is to better serve groups 
historically underrepresented in STEM fields. It defines groups underrepresented in STEM 
fields as Hispanics and Latinos, African Americans, American Indians, Alaska Natives, 
Native Hawaiians and Pacific Islanders, the economically disadvantaged, people with 
disabilities, and women and girls. 
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Figure 5: Percentage of Science, Technology, Engineering, and Mathematics (STEM) Education Programs That Tracked 
Certain Participant Characteristics in Fiscal Year 2016 
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Source: GAO analysis of questionnaire responses provided by agency officials on STEM education programs. | GAO-18-290 


In addition, 7 of the 13 administering agencies, such as the Department of 
Health and Human Services, reported that they tracked participation in 
fiscal year 2016 for at least two-thirds of their STEM education 
programs.°° Officials from the Department of Health and Human Services 


35 The seven agencies that reported via our questionnaire that they tracked the number of 
participants for at least two-thirds of their programs are the Departments of Agriculture, 
Commerce, Defense, Energy, Health and Human Services, and Interior, and the 
Environmental Protection Agency. 
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Conclusions 


said that the department maintains data for many of its STEM education 
programs in a database that captures individual participants’ demographic 
data, including race and gender, and aggregates such information for 
internal reporting. Officials also said they use this information to evaluate 
whether individual programs are meeting their goals of serving particular 
groups. Although we found that many agencies reported collecting data 
on participants in their STEM education programs, the Committee on 
STEM Education has not reported such information in its inventory, as 
required. 


Reporting information on the participation rates of women, 
underrepresented minorities, and persons in rural areas could help the 
Committee assess whether STEM education programs have broadened 
participation to groups historically underrepresented in STEM fields—a 
key goal of the Strategic Plan. Committee leadership said they measured 
progress toward this goal with general performance indicators, such as 
the number of women who earned STEM degrees, regardless of 
participation in federal programs, because such data were readily 
available. However, those performance indicators are influenced by 
various factors, including some external to federal STEM education 
efforts. For example, the number of women earning STEM degrees could 
be affected by broader economic factors or college enrollment trends, 
rather than the activities of the agencies. 


The federal government continues to invest billions of dollars annually in 
STEM education programs to enhance the nation’s economic and 
educational competitiveness. Since 2010, the federal portfolio of STEM 
education programs has evolved considerably. The Committee on STEM 
Education reported that, through its leadership and strategic planning 
efforts, it fostered coordination among agencies administering STEM 
education programs, which helped them implement the STEM Education 
Strategic Plan. Such efforts to encourage interagency coordination can 
help ensure efficient use of resources, particularly given the overlap of 
programs in the STEM education portfolio. 


The Committee on STEM Education and the Office of Science and 
Technology Policy have not fulfilled their responsibilities to review, 
document, and report performance information on STEM education 
programs. Reviewing performance assessments of the many programs in 
the federal STEM education portfolio is a vital management responsibility 
that could, for example, improve the Committee’s ability to disseminate 
information on promising practices or make recommendations that 
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Recommendations for 
Executive Action 


agencies can use to make well-informed decisions about designing or 
revising their programs. Further, documenting programs’ performance 
assessments in the Committee’s federal STEM education inventory and 
reporting the outcomes of recent assessments in the Office of Science 
and Technology Policy’s annual reports could enhance the availability of 
performance information. 


In addition, the Committee falls short in reporting required information on 
programs’ participation rates of women, underrepresented minorities, and 
persons from rural areas. Without such information, it is unclear whether 
the federal investment in STEM education is ultimately supporting its 
strategic goal of broadening participation to groups historically 
underrepresented in STEM fields. 


Moreover, as the Committee on STEM Education begins to implement its 
new responsibilities prescribed by the American Innovation and 
Competitiveness Act, its efforts to review programs’ performance 
assessments could improve its capacity to make well-informed 
recommendations to further enhance the portfolio of STEM education 
programs. 


We are making a total of four recommendations, including three to the 
Committee on STEM Education and one to the Office of Science and 
Technology Policy. Specifically: 


The leadership of the Committee on STEM Education should 
review performance assessments of federal STEM education 
programs and then take appropriate steps to enhance 
effectiveness of the portfolio, such as by sharing promising 
practices that agencies could use in designing or revising their 
programs. (Recommendation 1) 


The leadership of the Committee on STEM Education should 
improve public awareness of information on programs’ 
performance assessments by documenting program-level 
information on performance assessments in its federal STEM 
education inventory. (Recommendation 2) 


The leadership of the Committee on STEM Education should 


report required information on the participation rates of women, 
underrepresented minorities, and persons from rural areas in 
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Agency Comments 
and Our Evaluation 


federal STEM education programs that collect this information. 
(Recommendation 3) 


The Director of the Office of Science and Technology Policy 
should report the outcomes of programs’ performance 
assessments completed in the previous year in its annual report. 
(Recommendation 4) 


We provided a draft of this report to the National Science and Technology 
Council’s Committee on STEM Education and the Office of Science and 
Technology Policy for review and comment. These entities jointly 
provided written comments, which are reproduced in appendix IV, and 
technical comments, which we incorporated, as appropriate. They agreed 
with all four of our recommendations and noted initial strategies for how 
they would implement three of them. Regarding implementation of the 
fourth recommendation to report on participation rates of 
underrepresented groups in federal STEM education programs, they 
noted plans to examine confounding factors inhibiting the reporting of the 
information required under the COMPETES Act. Gaining insight on the 
challenges agencies face collecting this information is an important first 
step. However, to comply with the requirement of the COMPETES Act 
and help ensure programs reach populations historically 
underrepresented in STEM fields, we continue to believe that the 
Committee should report the participation rates of women, 
underrepresented minorities, and persons from rural areas in federal 
STEM education programs that collect this information. To do so, the 
Committee may also need to develop strategies to help agencies 
overcome some of these confounding factors. 


We are sending copies of this report to leadership of the Committee on 
STEM Education, and the Assistant Director of STEM Education at the 
Office of Science and Technology Policy, and the appropriate 
congressional committees. In addition, the report will be available at no 
charge on GAO's website at http://www.gao.gov. 
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If you or your staff should have any questions about this report, please 
contact me at (617) 788-0534 or emreyarrasm@gao.gov. Contact points 
for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. GAO staff who made key 
contributions to this report are listed in appendix V. 


OV Aadaw Ervey « Anew 
Melissa Emrey-Arras 
Director, Education, Workforce, and Income Security Issues 
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Appendix |: GAO’s Methodology for Program 
Identification and Data Collection 


How GAO Identified To identify the programs that should receive our questionnaire, we sought 
Federal Education input from the 13 agencies that administer federal science, technology, 
Programs on Science engineering, and mathematics (STEM) education programs. We provided 


“ : each of the agencies with our definition of a STEM education program 
Technology, Engineering, and asked agency officials to identify programs funded in fiscal year 2016 
and Mathematics that met this definition (see text box). We also asked agency officials to 
provide information on the status of the 209 STEM education programs 
we included in our previous report on STEM education programs. ' 
Specifically, we asked whether the programs were funded in fiscal year 
2016 and, if not, whether they were consolidated or terminated. 


1 GAO, Science, Technology, Engineering, And Mathematics Education: Strategic 
Planning Needed to Better Manage Overlapping Programs across Multiple Agencies, 
GAO-12-108 (Washington, D.C.: Jan. 20, 2012). 
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Appendix I: GAO’s Methodology for Program 
Identification and Data Collection 


Definition of Science, Technology, Engineering, and Mathematics (STEM) Education Program 


GAO defined “STEM education program” as a program funded by allocation or congressional appropriation. An organized set of 
activities was considered a single program even when its funds were also allocated to other programs. A STEM education program 
that met the definition had one or more of the following as a primary objective: 


- attract or prepare students to pursue classes or coursework in STEM areas through formal or informal education activities 
(informal education programs provide support for activities that offer students learning opportunities outside of formal schooling 
through contests, science fairs, summer programs, and other means; outreach programs aimed at the general public were not 
included); 


e attract students to pursue degrees (2-year, 4-year, graduate, or doctoral degrees) in STEM fields through formal or informal 
education activities; 


e — provide training opportunities for undergraduate or graduate students in STEM fields (this can include grants, fellowships, 
internships, and traineeships that are intended for students; general research grants that involve hiring a student for lab work 
were not considered a STEM education program); 


e attract graduates to pursue careers in STEM fields; 

e — improve teacher (preservice or in-service) education in STEM fields; 

e improve or expand the capacity of K-12 schools or postsecondary institutions to promote or foster education in STEM fields; and 
e conduct research to enhance the quality of STEM education programs provided to students. 


Programs designed to retain current employees in STEM fields were not included. Programs that fund retraining of workers to pursue a 
degree in a STEM field were included because these programs help increase the number of students and professionals in STEM fields 
by helping retrain non-STEM workers to work in STEM fields. Also included were health care programs that train students for careers 
that are primarily in scientific research, but not those that train students for careers that are primarily in patient care (e.g. those that 
trained nurses, doctors, dentists, psychologists, or veterinarians). 


Lastly, GAO considered STEM fields to include any of the following broad disciplines: agricultural sciences; astronomy; biological 
sciences; chemistry; computer science; earth, atmospheric, and ocean sciences; engineering; material science; mathematical 
sciences; physics; social sciences (e.g., psychology, sociology, anthropology, cognitive science, economics, behavioral sciences); and 
technology. 


GAO used this same definition of STEM education program in its 2012 report. However, in the current report, GAO explicitly specified 
astronomy and material science as STEM fields and also revised “mathematics” to be “mathematical sciences” based on feedback 
from agency officials. 


Source: GAO | GAO-18-290 


We reviewed the information agencies submitted and took steps to 
corroborate it, such as by reviewing program descriptions and budget 
documents. Based on our analysis of this information, we sent a web- 
based questionnaire to 198 programs (see table 1). 
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Appendix I: GAO’s Methodology for Program 
Identification and Data Collection 


Table 1: Changes Agencies Reported in the Federal Portfolio of Science, 
Technology, Engineering, and Mathematics (STEM) Education Programs between 
Fiscal Years 2010 and 2016 


Agency reported portfolio changes Number of 

programs 

Previously reported for fiscal year 2010 209 
Programs removed from the 2016 portfolio 

Terminated or consolidated 77 

No longer reported as STEM education 11 


Programs added to the 2016 portfolio 


Newly created 74 
Newly reported as STEM education? 3 
Identified for fiscal year 2016 and sent a questionnaire 198 


Source: GAO analysis of agency submissions on their STEM education programs. | GAO-18-290 


* We previously reported these 11 programs as STEM education for fiscal year 2010 because their 
administering agencies reported them as STEM education programs in fiscal year 2010. For this 
report, the administering agencies for these 11 programs did not report them as meeting the STEM 
education definition in fiscal year 2016. 


” Officials from the Department of Agriculture reported 2 programs as STEM education for 2016 that 
they had not reported previously for 2010, despite being funded then. Officials from the National 
Aeronautics and Space Administration reported 2 programs separately for fiscal year 2016 that they 
had previously reported together as one program for fiscal year 2010. Thus, we sent questionnaires 
to 3 programs newly reported as STEM education for 2016. 


To develop the questionnaire and collect the data, we used recognized 
survey design practices to enhance data quality. For instance, we ordered 
the questionnaire appropriately and ensured the questions were clearly 
stated and easy to understand. The questionnaire solicited information on 
federal STEM education programs, including programs’ objectives, 
intended groups served, services provided, STEM fields, and obligations. 
We did not conduct pretests because most of the questions were included 
in our prior questionnaire and had already been pretested.? 


On May 8, 2017, we sent an email announcing the online questionnaire to 
the officials responsible for programs identified as STEM education and 
also notifying them that the questionnaire would be activated that week. 
On May 10, 2017, we sent a second message to officials informing them 
that the questionnaire was activated and providing them with unique 
usernames and passwords. As necessary, we followed-up with program 


? We administered a similar questionnaire for our 2012 report on STEM education 
programs (GAO-12-108). 
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Appendix I: GAO’s Methodology for Program 
Identification and Data Collection 


officials by telephone and email. We collected responses through August 
31, 2017. 


Based on our analysis of the questionnaire responses and other 
information we received from program officials, we excluded 35 programs 
from our inventory. (See table 2 for a summary of those 35 programs and 
the reasons we excluded them.) 


Table 2: Programs Excluded from our Federal Science, Technology, Engineering, 
and Mathematics (STEM) Education Inventory 


Number 
Reason of programs 
Program did not have STEM education as a primary objective 20 
Program did not receive congressional allocation or appropriation 13 
in fiscal year 2016 
Program was reported twice by the agency 2 
Total 35 


Source: GAO analysis of information on federal STEM education activities. | GAO-18-290 


Nine of the 35 excluded programs had been reported by agency officials 
as STEM education programs in our previous report. In most cases (8 of 
9), we excluded these programs in this report because the programs did 
not include STEM education as a primary objective in fiscal year 2016. In 
the remaining case, we excluded the program because it was a 
component of another fiscal year 2016 STEM education program, and 
thus would be duplicative. We confirmed this information and the 
programs’ exclusion with the administering agencies. 


After we completed our analysis, we identified 163 programs as STEM 
education for fiscal year 2016. Programs officials responsible for all 163 
of these programs completed our questionnaire. We used standard 
descriptive statistics to analyze responses to these completed 
questionnaires. We also used recognized survey design practices to 
process and analyze data collected via the questionnaire. For instance, 
we performed automated checks to review the data and identify 
inappropriate answers. We also reviewed the data for missing or 
ambiguous responses and followed up with program officials when 
necessary to clarify their responses. We did not verify all responses since 
we had applied recognized survey design practices and follow-up 
procedures, and had determined that the data used in this report were of 
sufficient quality for the purposes of our reporting objectives. 
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Appendix Il: Federal Science, Technology, 
Engineering, and Mathematics (STEM) 
Education Programs and Reported Fiscal 
Year 2016 Obligations 


Table 3: Federal STEM Education Programs and Reported Fiscal Year 2016 Obligations 


Reported 

fiscal year 2016 
STEM education 
program obligations 


Agency Program (in dollars) 

National Aeronautics Minority University Research and Education Project (MUREP) 31,995,826 

Pune seem National Space Grant College and Fellowship Project (Space Grant) 39,999,988 
STEM Education and Accountability Project (SEAP) 24,844,939 

National Science Advanced Technological Education (ATE) 66,040,000 

Foundation Advancing Informal STEM Leaming (AISL) 62,500,000 
Alliances for Graduate Education and the Professoriate (AGEP) 8,000,000 
Cultivating Cultures for Ethical STEM (CCE STEM) 2,670,000 
CyberCorps(R): Scholarship for Service (SFS) 49,980,000 
Discovery Research PreK-12 (DRK-12) 84,300,000 
East Asia & Pacific Summer Institutes for U.S. Graduate Students (EAPSI) 1,380,000 
Education and Human Resources Core Research (ECR) 67,570,000 
‘Graduate Research Fellowship (GRF) Program si ast‘!!! 882,340,000° 
Historically Black Colleges and Universities Undergraduate Program (HBCU-UP) 35,010,000 
Improving Undergraduate STEM Education (IUSE) 104,770,000 
Inclusion across the Nation of Communities of Learners of Underrepresented 
Discoverers in Engineering and Science (NSF INCLUDES) 13,970,000 
International Research Experiences for Students (IRES) 5,930,000 
Louis Stokes Alliances for Minority Participation (LSAMP) 46,010,000 
National Science Foundation Research Traineeship (NRT) 55,980,000 
Research Experiences for Teachers (RET) in Engineering and Computer Science 6,140,000 
‘Research Experiences for Undergraduates (REU) i (assti(<‘C!;*;*;*;*;!;!~!~!!~!~!~. «92,720,000: 
Robert Noyce Teacher Scholarship Program 64,500,000 
STEM + Computing (STEM+C) Partnerships 64,370,000 
Tribal Colleges and Universities Program (TCUP) 14,010,000 

Nuclear Regulatory Integrated University Program (IUP) 15,102,000 

Commission Minority Serving Institutions Program (MSIP) 1,008,285 

Department of 

Agriculture 

Animal and Plant AgDiscovery Program 907,034 
Hera ene ection 1890 National Scholars Program 0? 
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Appendix Il: Federal Science, Technology, 
Engineering, and Mathematics (STEM) 
Education Programs and Reported Fiscal Year 
2016 Obligations 


Reported 

fiscal year 2016 
STEM education 
program obligations 


Agency Program (in dollars) 
National Institute 1890 Institution Teaching, Research and Extension Capacity Building Grants (CBG) 
of Food and Program 17,457,425 
Agriculture Agriculture and Food Research Initiative Education and Literacy Initiative Research 
and Extension Experiences for Undergraduate Fellowships 4,101,236 
Agriculture and Food Research Initiative Education and Literacy Initiative 
Professional Development for Secondary School Teacher and Education 
Professionals 2,000,000 
Agriculture and Food Research Initiative Fellowships (Predoctoral and Postdoctoral)” 11,417,025 
Agriculture in the Classroom 398,028 
Distance Education Grants Program for Institutions of Higher Education in Insular 
Areas 800,000 
Higher Education Challenge Grants Program 4,378,970 
Hispanic Education Partnership Grants 8,840,842 
Multicultural Scholars 936,873 
National Needs Fellowships 336,203 
Secondary Education, Two-Year Postsecondary Education and Agriculture in the K- 
12 Classroom Grants 852,300 
Resident Instruction Grants Program for Institutions of Higher Education in Insular 
Areas 1,000,000 
Women and Minorities in Science, Technology, Engineering and Mathematics Fields 
Grant Program (WAMS)? 382,650 
Department of 
Commerce 
National Institute NIST Summer Institute for Middle School Science Teachers 200,000 
of Standards and NIST Graduate Student Measurement Science & Engineering (GMSE) Fellowship 
Technology Program 640,000 
(NIST) : 
NIST Summer Undergraduate Research Fellowship (SURF) Program 1,700,000 
National Oceanic Bay Watershed Education and Training (B-WET) Program 7,952,700 
and Atmospheric Environmental Literacy Program 3,000,000 
Administration 
(NOAA) Educational Partnership Program with Minority Serving Institutions 14,431,000 
Ernest F. Hollings Scholarship Program 5,830,000 
National Sea Grant College Program (National Sea Grant College Program- 
Education Component) 904,061 
Teacher at Sea Program 600,000 
Department of 
Defense 
Air Force Air Force K-12 STEM Outreach (AFSTEM) 3,995,000 
National Defense Science and Engineering Graduate (NDSEG) Fellowship 43,905,926 
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Reported 

fiscal year 2016 
STEM education 
program obligations 


Agency Program (in dollars) 
Army Army Educational Outreach Program (AEOP) 10,220,356 
Navy Science and Engineering Apprentice Program (SEAP) * 

SeaPerch 57,000 
The Naval Research Enterprise Intern Program (NREIP) © 
Secretary of DoD STARBASE Program 25,000,000 
Defense National Defense Education Program (NDEP) Military Child Pilot Program 13,466,982 
National Defense Education Program (NDEP) K-12 5,512,106 
National Defense Education Program (NDEP) Science, Mathematics And Research 
for Transformation (SMART) 33,602,660 
Department of Developing Hispanic-Serving Institutions: STEM and Articulation Programs 92,425,366 
Education Graduate Assistance in Areas of National Need® 29,293,000 
Mathematics and Science Partnerships 152,717,000 
Minority Science and Engineering Improvement Program 9,648,000 
Research in Special Education® 44,775,891 
Research, Development, and Dissemination® 179,032,382 
Strengthening Predominantly Black Institutions‘ 13,905,000 
Upward Bound Math-Science 44,289,274 
Department of Advanced Vehicle Competitions 2,500,000 
Energy American Chemical Society Summer School in Nuclear and Radiochemistry 561,000 
BioenergizeME 300,000 
Collegiate Wind Competition 750,000 
Community College Internships (CCl) 1,000,000 
Computational Science Graduate Fellowship 11,500,000 
Environmental Management-Minority Serving Institution Partnership Program 
(MSIPP) 5,615,075 
Environmental Management Traineeship in Robotics 3,000,035 
Hampton University Graduate Studies 50,000 
Industrial Assessment Centers 7,000,000 
Integrated University Program (IUP) 4,980,486 
Mickey Leland Energy Fellowship Program 700,000 
Minority Educational Student Partnership Program (MEISPP) (summer interns) 933,900 
National Nuclear Security Administration-Minority Serving Institutions Partnership 
Program 16,500,000 
National Science Bowl (NSB) 2,900,000 
Office of Science Graduate Student Research (SCGSR) 2,500,000 
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2016 Obligations 


Reported 

fiscal year 2016 
STEM education 
program obligations 


Agency Program (in dollars) 
Race to Zero Student Design Competition 472,500 
Radiochemistry Summer School 300,000 
Science and Technology Workforce Development Initiative 1,000,000 
Science Undergraduate Laboratory Internships (SULI) 8,300,000 
Solar Decathlon 2,500,000 
Traineeship in Radiochemistry 3,000,000 
Visiting Faculty Program (VFP) 1,700,000 

Department of Health 

and Human Services 

National Aging Research Dissertation Awards to Increase Diversity 451,361 
hah as ea Blueprint Program for Enhancing Neuroscience Diversity through 

Undergraduate Research Education Experiences 2,333,036 
Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative: 
Short Courses in Computational Neuroscience 401,280 
Bridges to the Baccalaureate Program 9,549,894 
Bridges to the Doctorate 3,424,610 
Cancer Education Grants Program 10,771,449 
Cancer Research Education Grants Program-Research Experiences 912,361 
Center for Cancer Research Cancer Research Interns (Cancer Research Interns) 182,232 
Center for Cancer Research/John Hopkins University Master of Science in 
Biotechnology Concentration in Molecular Targets and Drug Discovery Technologies 266,976 
Courses for Skills Development in Biomedical Big Data Science 4,356,583 
Drug Abuse Dissertation Research 413,106 
Enriching the Hematology Research Workforce through Short-term Educational 
Experiences in Emerging Science Research Education Program Grant 311,644 
Graduate Partnerships Program 10,873,246° 
Initiative for Maximizing Student Development 22,442,998 
Initiative to Maximize Research Education in Genomics: Diversity Action Plan 2,995,000 
Innovative Programs to Enhance Research Training (IPERT) 6,761,954 
Intramural National Institute of Allergy and Infectious Diseases Research 
Opportunities 260,396 
Maximizing Access to Research Careers (MARC) U-STAR 17,484,652 
Medical Informatics Training Program 900,000 
Mental Health Research Dissertation Grant to Increase Diversity 70,725 
National Cancer Institute Predoctoral to Postdoctoral Fellow Transition Award (F99 
portion only) 1,366,103 
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Reported 

fiscal year 2016 
STEM education 
program obligations 


Agency Program (in dollars) 
National Institute of Allergy and Infectious Diseases Division of Intramural Research 
(DIR) Office of Training and Diversity (OTD) Sponsorship Program 561,778 
National Institute of Allergy and Infectious Diseases Division of Intramural Research 
(DIR) Summer Diversity Program 14,106 
National Institute of Diabetes and Digestive and Kidney Diseases Research 
Education Program Grants for Summer Research Experiences (R25) 1,200,000 
National Institute of Mental Health Mentoring Networks for Mental Health Research 
Education 786,789 
National Institute of Mental Health Research Education Mentoring Programs for 
HIV/AIDS Research 2,812,201 
National Institute of Mental Health Short Courses for Mental Health-Related 
Research Education 1,416,275 
National Institute of Neurological Disorders and Stroke Neuroscience Development 
for Advancing the Careers of a Diverse Research Workforce 1,020,046 
National Institute of Nursing Research Summer Genetics Institute 65,000 
National Institute on Aging Medicine, Science, Technology, Engineering and 
Mathematics: Advancing Diversity in Aging Research (ADAR) through 
Undergraduate Education 4,248,443 
National Library of Medicine Institutional Training Grants for Research Training in 
Biomedical Informatics and Data Science 12,672,914 
NIH Big Data to Knowledge (BD2K) Enhancing Diversity in Biomedical Data Science 673,730 
NIH Building Infrastructure Leading to Diversity (BUILD) Initiative (RL5 portion only) 10,480,823 
NIH Building Infrastructure Leading to Diversity (BUILD) Initiative (TL4 portion only) 16,427,440 
Postbaccalaureate Intramural Research Training Award Program 32,642,180° 
Postbaccalaureate Research Education Program (PREP) 11,193,892 
Research Education Grants for Statistical Training in the Genetics of Addiction 505,160 
Research Initiative for Scientific Enhancement 27,291,181 
Research Supplements to Promote Diversity in Health-Related Research 35,956,662 
Ruth L. Kirschstein National Research Service Award Institutional Research Training 
Grants (T32, T35, T90, TL1) 295,884,161 
Ruth L. Kirschstein National Research Service Award for Individual Predoctoral 
Fellows, including Underrepresented Racial/Ethnic Groups and Students from 
Disadvantaged Backgrounds 78,108,710 
Science Education Drug Abuse Partnership Award 483,000 
Science Education Partnership Award 18,541,000 
Short Courses on Mathematical, Statistical, and Computational Tools for Studying 
Biological Systems 2,416,584 
Short-Term Research Education Program to Increase Diversity in Health-Related 
Research 4,574,028 
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Reported 

fiscal year 2016 
STEM education 
program obligations 


Agency Program (in dollars) 
Short-Term Research Experience for Underrepresented Persons (STEP-UP; R25) 14,588,697 
Student Intramural Research Training Award Program 7,276,883° 
Summer Institute for Research Education in Biostatistics 1,470,346 
Summer Research Education Experience Programs 3,620,084 
Team-Based Design in Biomedical Engineering Education 615,518 
Training in Computational Neuroscience: From Biology to Model and Back Again 
(R90 portion only) 865,048 
Undergraduate Research Education Program (UP) to Enhance Diversity in the 
Environmental Health Sciences 645,215 
Undergraduate Scholarship Program for Individuals from Disadvantaged 
Backgrounds 2,249,743 
Undergraduate Summer Institutes in Kidney, Urologic and Hematologic Diseases 
(R25) 621,600 
Department of 
Homeland Security 
Science and Homeland Security STEM Summer Internship Program 500,000 
Technology Minority Service Institutions-Scientific Leadership Awards 3,600,000 
Directorate 
Minority Service Institutions-Summer Research Team 500,000 
Department of the 
Interior 
U.S. Geological | Education Component of the National Cooperative Geologic Mapping Program 
Survey (EDMAP) 419,438 
National Association of Geoscience Teachers (NAGT)-USGS Cooperative Summer 
Field Training Program 208,000 
Student Intern in Support of Native American Relations (SISNAR) 68,738 
Bureau of Indian Bureau of Indian Affairs, Office of Trust Services, Pathways Internship Program 475,082 
Affairs Science Post Graduate Scholarship Fund 2,450,000 
Department of 
Transportation 
Office of the University Transportation Centers Program 
Secretary of 
Transportation 67,700,000 
Federal Aviation Joint University Program 
Administration 250,000 
Federal Highway Garrett A. Morgan Technology and Transportation Education Program 332,150 
Administration National Summer Transportation Institute 2,620,293 
Summer Transportation Internship Program for Diverse Groups 1,300,000 
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Reported 

fiscal year 2016 
STEM education 
program obligations 


Agency Program (in dollars) 
Environmental Cooperative Training in Environmental Sciences Research 532,768 
Protection Agency Environmental Education Grants Program 3,306,600 
Environmental Research Training Grant (University of Cincinnati/EPA Research 
Training Grant) 175,000 
National Environmental Education Program 2,175,500 
People, Prosperity, and Planet Award: National Student Design Competition for 
Sustainability 223,795 
President’s Environmental Youth Award 80,000 


Source: GAO questionnaire of STEM education programs. | GAO-18-290 


Note: Amounts obligated for each program for fiscal year 2016 were reported to us by program 
officials in response to our questionnaire. We did not independently verify this information. 


* This program has not obligated any funds to the public. It is a share-cost program supported by 
Department of Agriculture agencies. 


Department of Agriculture officials reported two programs as STEM education for 2016 that they had 
not reported previously for 2010, despite being funded then. 


° Department of Defense officials reported a combined obligation amount for these two programs: 
$2,216,482 


. According to Department of Education officials, these programs also support a wide variety of non- 
STEM related fields and activities. 


° According to Health and Human Services officials, obligations for these programs are estimates. 
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of Selected COMPETES Act Provisions to 
Coordinate Federal Science, Technology, 
Engineering, and Mathematics Education 


Table 4: Current Implementation Status of Selected COMPETES Act Provisions to Coordinate Federal Science, Technology, 
Engineering, and Mathematics (STEM) Education 


COMPETES Act provisions Implementation status 
1. The Office of Science and Technology Policy should establish a committee comprised of specified 
agencies that have STEM education programs and activities. @ 


Mandated responsibilities of the Committee: 


2. Coordinate the STEM education activities and programs of the federal agencies. 


3. Coordinate STEM education activities and programs with the Office of Management and Budget. 


4. Encourage the teaching of innovation and entrepreneurship as part of STEM education activities. 


5. Ensure that STEM education activities and programs across the agencies are not duplicative. 


6. Develop and implement a 5-year STEM education strategic plan, update every 5 years. 


Elements required in the 5-year strategic plan: 


a. Annual and long-term objectives. 


b. Common metrics to assess progress toward achieving the objectives. 


c. Approaches each participating agency will take to assess the effectiveness of its STEM 
education programs. 


d. The role of each agency in supporting programs and activities designed to achieve the annual 
and long-term objectives in the strategic plan. 


7. Establish, periodically update, and maintain an inventory of federal STEM education programs and 
activities, including documentation of assessments of program effectiveness and the participation 
rates of women, underrepresented minorities, and persons in rural areas. 


 —7 |\@'0 0 @ 


Mandated responsibilities of the Office of Science and Technology Policy: 


8. Encourage and monitor the efforts of the participating agencies to ensure the strategic plan is 
developed and executed effectively. 


9. Transmit an annual report to Congress on the 5-year STEM education strategic plan. 


Elements required in the Office of Science and Technology Policy’s annual report: 


10. A description of the STEM education programs and activities for the previous and current fiscal 
years, and any proposed programs under the President’s budget request. 


11. The funding levels for the STEM education programs and activities for each participating federal 
agency for the previous fiscal year and under the President’s budget request. 


12. An evaluation of the levels of duplication and fragmentation of federal STEM education programs and 
activities. 


13. Progress made in implementing the 5-year plan, including a description of the outcome of any 
program assessments completed in the previous year and any changes made to that plan since the 
previous report. 


7 |@@e\e 
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COMPETES Act provisions Implementation status 


14. How the participating federal agencies will disseminate information about federal resources for STEM @ 
education practitioners. 


Legend: @ = Fully implemented: actions taken fully addressed the provision 


q) = Partially implemented: actions taken addressed some, but not all, parts of the provision 
Source: GAO analysis of efforts to implement provisions of the COMPETES Act based on document review and agency input. | GAO-18-290 
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Appendix IV: Comments from the Office of 
Science and Technology Policy and the 
Committee on Science, Technology, 
Engineering, and Mathematics Education 


EXECUTIVE OFFICE OF THE PRESIDENT 
NATIONAL SCIENCE AND TECHNOLOGY COUNCIL 


WASHINGTON, D.C. 20502 


Thursday, March 1, 2018 


Melissa Emrey-Arras 

Director, Education, Workforce, and Income Security 
U.S. Government Accountability Office 

441 G Street, NW 

Washington, DC 20548 


Dear Ms. Emrey-Arras, 


On behalf of the leadership of the National Science and Technology Council (NSTC) and its 
Committee on Science, Technology, Engineering, and Mathematics Education (CoSTEM), I 
appreciate the opportunity, as Senior Policy Advisor and Assistant Director for STEM Education 
at the Office of Science and Technology Policy (OSTP), to convey this joint response to the 
recommendations in the U.S. Government Accountability Office (GAO) draft report, “Science, 
Technology, Engineering, and Mathematics Education: Actions Needed to Better Assess the 
Federal Investment” (GAO-18-290). STEM education is an urgent priority for COSTEM, OSTP, 
and the Trump Administration. In September, 2017, President Trump signed a Presidential 
Memorandum to boost investment in STEM education and computer science, which was 
immediately followed by pledges for matching investment by private industry. We look to 
continue the momentum. 


We are proud of the significant progress achieved in Federal STEM education, programming, 
coordination, and national leadership since the America Creating Opportunities to Meaningfully 
Promote Excellence in Technology, Education, and Science (COMPETES) Reauthorization Act 
of 2010. Fragmentation has been reduced and cross-agency partnerships have coalesced. Inter- 
and intra-agency human and fiscal resources are more focused on Congressional and 
Administration priorities. Promising practices in student retention, educator professional 
development, community engagement, broadening participation, and measurement of gains now 
permeate the Federal STEM education community, improving services, outcomes, and 
evaluation. 


Documentation of these gains has been catalogued annually by the Progress Report on 
Coordinating Federal Science, Technology, Engineering, and Mathematics Education delivered 
to Congress in compliance with the COMPETES Act. We appreciate that GAO captured the 
ongoing progress in Appendix III, designating twelve selected COMPETES Act provisions as 
“fully implemented” with two provisions designated as “partially implemented.” 


We are also pleased to highlight four significant developments that underscore important 
progress in recent months: 

e Under the auspices of CoSTEM through FC-STEM, a cross-agency STEM Education 
Evaluation Working Group, along with STEM education agency evaluators, convened in 
early 2018 to examine STEM education investment area evaluation plans and highlight 
promising practices for assessment, of which there are many. 
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Over the course of late 2017 and early 2018, OSTP, in partnership with Federal 
Coordination in STEM Education Committee (FC-STEM), a subcommittee of CoSTEM, 
convened each CoSTEM Interagency Working Group around the five Priority Investment 
Areas of the 2013-2018 Federal STEM Education 5-Year Strategic Plan to intake 
assessment information and set a course for the future. 

OSTP, in collaboration with the Office of Management and Budget (OMB) and FC- 
STEM, have designed and distributed a data-gathering instrument to catalogue gains on 
CoSTEM priorities across the 13 agencies comprising CoSTEM. Alongside OMB’s 
annual CoSTEM Program Inventory, information from this survey will allow 
comprehensive analyses for continued improvement of Federal STEM education 
coordination. 

In recent months, the Office of Science and Technology Policy welcomed strong 
leadership at the executive director position for the National Science and Technology 
Council and appointed a STEM education expert as senior policy analyst who brings 
three decades of experience and proven results. OSTP is well positioned to advance the 
Administration’s priorities, assessment and evaluation, inter-agency communication, 
reporting, and strategic planning for the future, in partnership with the STEM education 
leadership across the Federal government and with the U.S. Congress. 


We have reviewed the “Recommendations for Executive Action” (page 20), and with expanded 
capacity now in place at OSTP to support CoSTEM, we will move forward to put processes and 
strategies in place to consider how to address each one in a meaningful way over the coming 
months. Regarding program effectiveness recommendations: 


CoSTEM, in concert with agency evaluation experts and OMB performance staff, will 
focus on evaluation of the very diverse portfolio of STEM education investments across 
agencies and how to best review and report on STEM education assessment. 

CoSTEM will improve public awareness about Federal STEM investments by 
synthesizing highlights and trends about program-level performance assessment and 
impacts when possible. 

The Director of OSTP will include reporting on the available outcomes of programs’ 
performance assessments completed in the previous year, subject to reliable availability, 
in the annual report. 


Regarding the reporting of participation rates of underrepresented subpopulations: 


We embrace the imperative to deliver return-on-investment information, especially 
regarding the engagement of underrepresented groups, to STEM stakeholders across 
government and beyond. CoSTEM intends to examine confounding factors such as 
privacy law, self-report reliability, definitional challenges (e.g., “rural” and 
“Participation’’) inhibiting the fulfillment of this provision of the COMPETES Act. 


The Office of Science and Technology Policy and the Committee on STEM Education of the 
NSTC appreciate GAO’s shared commitment to advancing STEM education programs and better 
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preparing the nation’s youth. We have an exciting opportunity to make great strides in 
strengthening and improving Federal STEM activities to ensure future generations of students 
have the tools they need to succeed in an ever-evolving workforce, and that the United States has 
the talent necessary to maintain our global leadership in science and technology. It is imperative 
that our citizens young and old have the opportunity to develop the skills necessary to fill 21* 
century jobs. We look forward to continued collaboration. 


Sincerely, 


plied 


Jeff Weld, Ph.D. 

Senior Policy Advisor and Assistant Director, STEM Education 
Office of Science and Technology Policy 

Executive Office of the President 
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is reflected in its core values of accountability, integrity, and reliability. 


a ee ee 
“at : The fastest and easiest way to obtain copies of GAO documents at no cost is 
Obtaining Copies of through GAO’s website (https:/Awww.gao.gov). Each weekday afternoon, GAO 
GAO Reports and posts on its website newly released reports, testimony, and correspondence. To 
. have GAO e-mail you a list of newly posted products, go to https:/Awww.gao.gov 
Testimony and select “E-mail Updates.” 
Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and 


distribution and depends on the number of pages in the publication and whether 
the publication is printed in color or black and white. Pricing and ordering 
information is posted on GAO’s website, https://www.gao.gov/ordering.htm. 


Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 


Orders may be paid for using American Express, Discover Card, MasterCard, 
Visa, check, or money order. Call for additional information. 


: Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 
Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 
Visit GAO on the web at https:/Awww.gao.gov. 


To Report Fraud, ene 
Waste and Abuse in Website: https:/Awww.gao.gov/fraudnet/fraudnet.htm 


Automated answering system: (800) 424-5454 or (202) 512-7470 
Federal Programs a ee ny 


: Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, 
Congressional U.S. Government Accountability Office, 441 G Street NW, Room 7125, 
Relations Washington, DC 20548 
Public Affairs Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 


U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548 


: : James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707 
Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814, 


External Liaison Washington, DC 20548 


ow 


%a¢ 


Please Print on Recycled Paper. 


